Data Led Growth project | PocketFM

Overview

• Increasing Conversion% by increasing Activation%, number of users who listen to 1 hour of the show, "God Eye". Aim is to do this by increasing 0-95% completion rate of the first 5 episodes, which together add up to 1 hour. It is to be achieved by re-writing each episode by increasing stakes in the first 3 seconds of the episode and improving cliffhangers (which will impact 75%-95% completion of the episode)

Hypothesis

• If Activation% increases by 10%, then Conversion% will increase by 7%, because more users will be entering the conversion funnel. This show already has high conversion and high retention, but a low activation%

Goal

• Increasing 0-95% completion rate of the first 5 episodes by increasing 3 second play% (first 3 seconds played of the episode) and 75%-95% completion rate (listened device% of the last quadrant of the episode)

Success Metrics

• Best Case: 0-95% completion is above 3%
• Worst Case: 0-95% completion of the re-written episode is better than the current version live on app
• Health Metric: Cost Per Install should be less than $10

Experiment Design

• Testing a re-written episode 1 with the following changes:
• • new opening, to increase 3 Second Play% (15%+),
  • heightened cliffhangers, to increase 75%-95% completion (90%+),
  • reducing irrelevant information and increasing action.
• To increase 0-95% completion (3%+) while maintaining a healthy CPI (<$10).
• Supporting Data:
• • As a part of another experiment, we tested Activation% of the existing 1 hour of the show as an ad on meta (to get quicker results on the experiment - show has low LDAU on app)
  • Adset has a high CPI, a low 3 second play%, and a poor 0-25% completion.
  • The first episode makes up 18% of the adset. That's why we thought we'd approach it episode-wise. These episodes can then become assets for acquisition in the future.
• Variation Design:
• • Control: App As-is Episode 1
  • Variation A: Re-written Episode 1
  • Variation B: Re-written Episode 1
  • Same Budget, Same Ad controls, run simultaneously to remove CPM variability.
• Audience and Sample Testing:
• • Same Ad controls.
  • Sample Size to be around ~25k impressions on the adset and 20+ installs for all variants

Post Experiment

• Result:​
• • Variation A has the lowest 3 second play, but has the highest 0-25%, leading to the highest 0-95% completion rate. Not to mention, the lowest CPI (the lowest we've ever gotten with a promo for this show)
• Release Design
• • Scale Variation A to see if the CPI holds.
• Next Steps:
• • Re-write the first few seconds to increase 3 second play.
  • Re-write the second quadrant of the episode.

​

​

​

​